The completeness of adverse drug reaction reports in South Africa: An analysis in VigiBase®

Background Spontaneous reporting is regarded as a cornerstone of pharmacovigilance (PV) but presents many limitations, including varying quality and completeness of information, which is essential for causality assessment. Aim This study aimed to evaluate the completeness of adverse drug reaction (ADR) reports in South Africa based on the vigiGrade completeness score. Setting The South African Health Products Regulatory Authority (SAHPRA). Methods A cross-sectional, descriptive study of all reports received by SAHPRA and submitted to VigiBase® in 2017 was conducted. A report with a vigiGrade score > 0.8 is considered well-documented. Results The mean completeness score for the 8438 reports received was 0.456 (s.d. = 0.221). Only 11.3% of reports had a completeness score > 0.8. The completeness of reports submitted by consumers professionals did not significantly differ from reports by physicians, pharmacists or other healthcare professionals (d ≤ 0.2). Reports of reactions that resulted in death (M = 0.572, s.e. = 0.007), disability (M = 0.491, s.e. 0.033) or were life threatening (M = 0.474, s.e. = 0.013) had a medium to large practically significant effect (0.5 ≥ d ≤ 0.8) on the completeness score compared with reports of congenital anomaly (M = 0.348, s.e. = 0.089). Conclusion The completeness of reports submitted by consumers is comparable to those submitted by healthcare professionals. The completeness of reports was low and multiple measures to improve reporting are recommended. Contribution This study describes the completeness of ADR reports in South Africa and the results can be used to improve training.


Introduction
Pharmacovigilance (PV) is defined as 'the science and activities relating to the detection, assessment, understanding and prevention of adverse drug reactions (ADRs) or other drugrelated problems'. 1 Once approved for marketing, drugs are continuously monitored for ADRs to ensure a favourable risk-benefit ratio. Post-marketing safety monitoring mainly occurs via spontaneous reporting of ADRs by healthcare professionals (HCPs) or consumers to either the marketing authorisation holder (MAH) or the local regulatory authority. 2 According to Moore et al., spontaneous ADR reports were the primary source of scientific information used by the Food & Drug Administration (FDA) to make regulatory decisions during 2009 and formed the basis of 57% new regulatory actions and 76% new boxed warnings. 3 Further analysis by Lester et al. indicated that spontaneous reports were the typical evidence source contributing to label changes by the FDA in 2010. 4 The three most prevalent evidence sources that contributed to a label change were spontaneous reports (52%), clinical trials (16%) and pharmacokinetic studies (11%). 4 Even though spontaneous reporting is a cornerstone of PV, with many significant strengths, including early detection of safety alerts, it also has many limitations. These limitations include low reporting rates, varying quality, completeness and accuracy of the provided information. 5 The incompleteness of spontaneous reports poses a great concern because of causality assessment, 'the process during which the level of probability between the ADR and the role of the suspect drug as the causative agent is determined', depends on this parameter. 6 The quality of the information provided in case reports influences the strength of the causal relationship between the suspect drug and the reaction. 7 In 1965, the English epidemiologist Sir Austin Bradford Hill (1897-1991) established a group of minimal criteria necessary to provide adequate evidence of a causal relationship between incidence and a consequence. The Bradford Hill criteria, also known as Hill's criteria for causation, assess causality from multiple information sources using the following parameters: 'strength of association, temporality, consistency, theoretical plausibility, coherence, specificity in the causes, dose-response relationship, experimental evidence and analogy'. 8 Bergvall et al. have further indicated that the important information required for causality assessment leading to signal detection includes time-to-onset of the ADR, patient age and sex, outcome and indication for treatment. 9 Durrieu et al. revealed that only 12.7% of reports submitted by general practitioners were classified as well-documented and concluded that it is essential to promote quality data to optimise signal detection. 10 Sánchez-Sánchez et al. evaluated the completeness of suspected ADR reports submitted to the Mexican National Pharmacovigilance Centre in 2008 and indicated that most reports contained incomplete information. About 40% of the reports contained the date of suspected ADR; however, treatment start dates were unknown, making it impossible to determine the time-toonset of the reaction. 11 The Uppsala Monitoring Centre (UMC), the World Health Organization (WHO) collaborating centre for international drug monitoring, is responsible for signal detection and dissemination of drug safety issues. Therefore, the UMC holds and maintains the VigiBase ® (Uppsala, Monitoring Centre, Uppsalla, Sweden), the WHO global database of individual case safety reports (ICSRs). 12,13 Following the low levels of completeness of ICSRs submitted by regulatory authorities to VigiBase ® , a tool used to measure the level of completeness of ICSRs in VigiBase ® , known as vigiGrade, was developed by the UMC. 9,14 The Uppsala Monitoring Centre automatically calculates the vigiGrade completeness score for each report committed to VigiBase ® and reports with a completeness score > 0.8 are considered well-documented. 9 National pharmacovigilance centres can view the completeness score per report through VigiLyze ® (Uppsala Monitoring Centre, Uppsala, Sweden), the search and analysis software that enables exploration of VigiBase ® . Bergvall et al. have reported an average completeness score of 0.45 for all reports in VigiBase ® , while only 13% of the reports were considered well-documented. 9 Measuring and communicating the completeness to PV centres is the first step to help improve the quality of reports. 9 Having been a member of the WHO Programme for International Drug Monitoring (WHO-PIDM) since 1992, 15

Study design
A cross-sectional, descriptive study of all ICSRs received by the SAHPRA and submitted to VigiBase ® during 2017 was conducted.

Setting
The study was conducted in the Pharmacovigilance Unit of the SAHPRA.

Study population
Adverse drug reaction reports were captured into the VigiFlow ® system, which supports the domestic collection and processing of ICSRs, before being committed to the VigiBase ® . For MAHs, reporting of serious ADRs is mandatory, and these reports should be sent to SAHPRA within 15 calendar days of receipt. 16 Reports were received from different reporter categories as per their qualifications, namely physicians, pharmacists, consumers or non-HCPs, lawyers and other HCPs (e.g. nurses). VigiLyze ® exports do not indicate the sender of the report (i.e. MAHs, clinical trials, HCPs or non-HCPs) or the report reporting method (e.g. Med Safety App, e-reporting or paper form).

Data collection
The ICSRs were extracted from VigiBase ® on 11 June 2019 with the search and analysis software known as VigiLyze ® . VigiLyze ® exports contain information about each case report as well as complete information on medications and the ADR. Demographic variables of the patients included age and sex. Age groups: neonates (0-27 days), infants (28 days -23 months), children (2-11 years), adolescents (12-18 years), adults (19-64 years) and elderly (> 65 years). The country of origin for all reports analysed was South Africa.
The seriousness of the reactions was classified as resulting in death, life-threatening, disabling, hospitalisation or prolongation of hospitalisation, congenital anomaly and other medically important event. The outcome of the reaction was categorised as died, recovered, not recovered, recovering and unknown.
The suspect and concomitant medicines were classified according to the Anatomical Therapeutic Chemical (ATC) classification system, 17 in which the 'active substances are divided into different groups according to the organ or system on which they act and their therapeutic, pharmacological and chemical properties'. Drugs are classified in groups at five different levels, and in this study, medicines were evaluated according to level 1 within 14 main ATC groups. Adverse drug reactions were classified according to the Medical Dictionary for Regulatory Activities (MedDRA ® ) preferred term and System Organ Class. 18 The start date of the drug and the onset date of the reaction were used to calculate the time-to-onset of the reaction. The completeness score for each report is calculated by UMC and can be viewed in VigiLyze ® . The vigiGrade completeness score is restricted to important information for causality assessment and is expected to be present on most reports. 9 The vigiGrade completeness score evaluates the dimensions of the report rather than the specific elements included in it. Three levels of importance are identified, namely: [E]ssential (information without which reliable causality assessment is impossible), important (information without which reliable causality assessment is very difficult), and supportive (information that is valuable but without which reliable causality assessment can still be performed). 9 If a dimension is not reported, the completeness score suffers a penalty. Imprecise information is penalised less compared with the total omission of information. The more important the information is in the clinical assessment of a drug reaction relation, the higher the penalty factor. 9 The vigiGrade completeness score (C) starts at 1 for reports containing all important variables for causality assessment. The vigiGrade penalties are defined as indicated in Table 1. 9 vigiGrade classifies a report with a final score of C ≥ 0.8 as well-documented. 9 This limit allows for lack of information on two supportive variables, but with a penalty of 50% on the essential variables and 30% on important variables. 9 The vigiGrade completeness score (C) in categories < 0.5, 0.5 ≥ 0.8 and > 0.8 has been reported.

Data analysis
The Medical Dictionary for Regulatory Activities (MedDRA ® version 22.0) de-duplicated data set was exported to Microsoft Excel and was analysed using the Statistical Package for Social Sciences (SPSS ® ) version 25.0. 19 Data were first analysed using descriptive statistics, where frequencies and percentages were calculated for all variables. Measures of central tendency for continuous variables were displayed as means and corresponding standard deviation or error of the mean. The one-way analysis of variance (ANOVA) was used to compare the continuously distributed mean completeness scores between different categories of each variable analysed. All statistical tests were conducted at the 5% significance level, and Cohen's d was calculated as a measure of effect size. Cohen's d guideline: small effect ≤ 0.2; medium effect > 0.2 and < 0.5; large effect ≥ 0.5 and < 0.8; practical significant ≥ 0.8.

Ethical approval
This study was approved by the North-West University Health Research Ethics Committee (No. NWU-00012-19-S1) and goodwill permission was obtained from the South African Health Products Regulatory Authority (SAHPRA). This study was performed in line with the principles of the Declaration of Helsinki.

Results
South African Health Products Regulatory Authority received 8438 ICSRs that were committed to VigiBase ® from  Time-to-onset of the ADR, which is a crucial factor needed for causality assessment, could not be calculated for 52.82% (n = 10795) of suspected drug-reaction pairs. The vigiGrade completeness score applies a 50% penalty if time-to-onset is missing in the report. The ADR outcome was missing on 10.94% (n = 923) of the reports and suffered a 30% vigiGrade penalty. Where the event outcome was reported, it was reported as unknown on 63.32% (n = 5343) of the reports. The indication for using the suspected drugs was missing on 16% (n = 3278) of reports, resulting in a 30% vigiGrade penalty. Among the 8348 reports analysed, the age and sex were incomplete in 2097 (24.8%) and 421 (4.9%) reports, respectively, and suffered a 30% vigiGrade penalty. The drug dose was incomplete for 50.30% (n = 15 011) of the drugreaction pairs and suffered a 10% penalty. Concerning 'action taken with the drug', 15% (n = 3075) of reports did not have this variable completed.

Seriousness of the reports
The suspected reaction was reported as fatal in 12.47% (n = 1052) ( Table 5)    Outcome Table 6 contains the frequencies and completeness of ADR according to different outcomes.
The one-way ANOVA test showed a statistically significant difference in the completeness score for reports with different outcomes F (5, 8432) = 624.42, p < 0.001. The outcome of the ADR was missing in 10.94% (n = 923) of the reports and suffered a 30% vigiGrade penalty. This had a practically significant effect (d ≥ 0.8) on the mean completeness score when compared with reports where the outcome of the ADR was specified. On 63.32% (n = 5343) of the reports, the outcome was classified as unknown, and there was a large, practically significant difference (d ≥ 0.8) between the mean completeness score for reports that were classified as unknown (M = 0.394, s.e. = 0.003), compared with all the other outcome categories.

Anatomical Therapeutic Chemical classes
The reports contained 20 438 different suspected drugreaction pairs for drugs belonging to different ATC classes (Table 7).
There was a statistically significant difference in the completeness score for reports according to the different ATC classes of the suspected drugs F (13, 8424) = 48.727, p ≤ 0.001. Reports containing suspected drugs that belong to ATC Class B (blood and blood-forming products) had the highest mean completeness score (M = 0.592, s.e. = 0.008), and there was a practically significant difference between the completeness score of Class B compared with Classes D, G, H, M, N, P, R and V (d ≥ 0.8).

Discussion
Lack of essential information in ADR reports has been an obstacle in the quality management of signal detection, rendering ADR reports futile. 7,10 Several tools have been developed and used to measure and enhance the completeness of ADR reports. These include the vigiGrade completeness score, EudraVigilance feedback report, clinical documentation tool, and the quality of ADR reports algorithm. 9,20 In this study, the authors used the vigiGrade completeness score to assess the completeness of ADR reports received by the SAHPRA during 2017, as it is the tool that was designed to measure the level of clinically relevant information in ICSRs in VigiBase ® . The main use of vigiGrade is to communicate data quality to member countries of the WHO-PIDM. 21 Our results indicated that only 11.3% (n = 953) of the reports analyzed are considered to be well-documented (C > 0.8). This is in line with the findings of Bergvall et al., who used VigiGrade to calculate the completeness score for 3.3 million reports in VigiBase ® between 2007 and 2012 and found that only 13% of global reports in VigiBase ® were considered welldocumented and achieved a completeness score > 0. 8. 9 This indicates that more than half of the reports received did not have the essential and important information required for signal detection. The mean completeness score of the reports analyzed in this study is 0.456 (s.d. = 0.22), which is aligned with the findings made by WHO. 9 This indicates poor quality of reports, which may be explained by the fact that the study included both serious and non-serious reports. However, there are countries such as Japan where 49.5% of reports had a vigiGrade completeness score above 0.8 and were classified as well-documented reports.

Completeness of reports for different categories of primary reporters
Physicians reported the most (38.6%), followed by consumers or non-HCPs (28.6%) and other HCPs (25.78%), while pharmacists submitted only 4.3% of reports. Aagaard et al. reported that 75% of reports came from physicians compared with 13% and 11% for other HCPs and consumers, respectively. 22 Thiessard et al. reported 91% of reports coming from physicians and 5% from pharmacists. 23 In this study physicians submitted the highest number of reports, but when compared with the other studies, it was not such a convincing majority that may suggest underreporting by South African physicians. Physicians are exposed to serious ADRs in hospital settings and have a clinical background about patients' medical conditions and therefore they might have higher reporting rates when compared with other reporter types. However, the reporting rates of serious ADRs per primary reporter category were not determined. In this study, the completeness of reports submitted by consumers or non-HCPs did not have a practically significant difference compared with reports submitted by physicians, pharmacists and other HCPs (p ≤ 0.001; d ≤ 0.2). This indicates that the completeness of reports from consumers is comparable to those submitted by HCPs. Rolfes et al. had previously indicated that HCPs and consumers reported clinical information on a comparable level. 24 A total of 61% of the reports submitted by physicians had a completeness score < 0.5. Only 9.6% of their reports were considered welldocumented, whereas 11.1% of the reports submitted by consumers or non-HCPs are well-documented. However, this finding is of interest as it is expected that physicians should submit reports with more clinical data than consumers because they have access to patients' complete profiles and medical training.

Missing information in the case reports
Time-to-onset is considered the most essential information in causality assessment because it indicates the temporal relation between the reaction and the suspect drug. Therefore, its omission in the reports accounts for a 50% vigiGrade penalty. The crucial information to calculate the time-to-onset of the reaction was not provided for 52.8% of the drug-reaction pairs, which means it is impossible to evaluate the causal relationship between the drug and the reaction and possibly detect safety signals from these reports. These findings correspond with a study conducted on the Japanese Adverse Event Report database using VigiGrade ® in which time-to-onset could not be calculated in 41.2% cases received from the pharmaceutical companies. 25 Time-to-onset is determined based on the start date of the drug and the onset date of the reaction. The start date of the suspected drug was missing in 52.6% of cases, while the onset date of the reaction was missing in 5.2% of cases. Lack of awareness, particularly in maintaining ADR quality reporting and complacency by HCPs in accepting standards for ADR reporting, maybe some of the reasons behind 52.8% where time-to-onset cannot be determined. 26 Other factors that can explain why the onset date of the drug is omitted is that patients cannot remember when they started their treatment, while HCPs may not have access to patient records at the time of reporting. Therefore, it is recommended to use an electronic health records system (EHRS) where these data are captured and accessed when required. The use of EHRS will eliminate recall bias by patients. The use of an EHR system has been studied and confirmed to be an additional source of information in characterising drug safety. 27 However, in a systematic review by Katatura and Cilliers the authors identified social, technical and environment barriers to the implementation of EHRS in African countries, which included a 'lack of supporting infrastructure, user training and commitment, political influence or strategy, legislation and regulations and the lack of a framework for implementation and management of EHRs'. South Africa will have to address these barriers to establish a EHRS to track and manage patients in the planned National Health Insurance scheme. 28 To our knowledge ADR reporting has not been built into any dispensing or practice software used in South Africa.
Information regarding the dose of the medicines was missing in 50.3% of cases, while indication for the suspect drugs was missing in 16% of the drug-reaction pairs. As highlighted by Brajovic et al., knowing the reason for taking the suspect drug is an important factor in determining the significance of the reaction. 27 Action taken with the suspect drug was missing in 15% of drug-reaction pairs, while 10.9% of reports did not report the outcome of the reaction. Where the outcome was reported, it was classified as unknown on 63.32% of the reports. There was a practically significant difference between mean completeness scores of reports that indicated outcome as unknown or missing compared with reports with other outcome categories (d ≥ 0.8). If the outcome is not specified on reports or is reported as unknown, HCPs do not follow up with patients who suffer ADRs. The outcome information is one of the fundamental parameters used in causality assessment. 9 Information on the outcome of the reaction and the action taken with the suspect drug strengthens the relationship between the suspect drug and the reaction. The lack of these parameters suggests a lack of knowledge by reporters regarding ADR management and the importance of different parameters included in ADR reporting forms. Awareness and skill development programmes on the promotion of quality ADR management and reporting need to be considered to improve the quality of ADR reports. Furthermore, follow-up of patients by HCPs needs to be equally addressed to ensure that complete information is sought from the patients.
This study revealed that 55.9% of the reactions were serious, of which 12.5% were fatal. This corresponds with a Danish study, which indicated that 52% of their reports were serious. 22 In line with the findings by Durrieu et al., results of this study have indicated an association between the seriousness of the reaction and the completeness of the report. 10 If the reaction was fatal, life-threatening or resulted in disability, it had a medium to large practically significant effect on the completeness scores of the reports (p < 0.001; 0.5 ≥ d ≤ 0.8). This indicates that reporters are more concerned about serious reactions, ensuring that reports concerning these reactions are well-documented. Serious cases are often hospitalised, and it is expect that they will have better followup and that reporters will have more complete records available while reporting.
A statistically significant difference in completeness score for reports of different ATC classes (p < 0.001) was observed. Suspect drugs belonging to ATC class B (blood and bloodforming products) contributed 6.5% of the total number of drugs reported and had the highest mean completeness score of 0.59 compared with the other groups (p ≤ 0.001); (d ≥ 0.8).
Blood and blood-forming agents are used to treat lifethreatening medical conditions such as stroke, which may explain why these reports have a higher completeness score than other ATC classes. This may further strengthen the association between the seriousness of the reaction and the completeness of the report.
Age and sex are considered important parameters in causality assessment because they indicate whether the suspect drug was prescribed appropriately or not. We found that the completeness score of reports improved with age, similar to a study conducted on the Saudi Food and Drug Authority PV system. 29 The completeness score for reports of adults and the elderly patients was higher than those of infants and neonates, and the difference has a medium to large practical significance (p < 0.001; 0.5 ≥ d ≤ 0.8). This might be because adults and elderly patients can clearly explain their experiences compared with infants and neonates who rely on the caregiver. Furthermore, 52% of the reports were for adults, followed by elderly patients with 11%, while reports for infants and neonates were about 3% and less than 1%, respectively. Increasing co-morbidities and polypharmacy may have contributed to an increased number of reports received for adults and elderly patients.

Strengths and limitations
This was the first study that has been conducted to evaluate the completeness of South African ADR reports submitted to VigiBase®. It is important that HCPs are made aware of the completeness of reports and how it impacts the ability of the SAHPRA to conduct causality assessment and detect signals to ultimately improve patient safety. The results of this study can be utilised to improve training of reporters and technical staff who work in the PV unit and are responsible for data capturing and data verification.
However, the present results are only for reports received in a 1-year study period, and it is not intended to be generalised beyond South Africa. A follow-up study should be conducted to evaluate ADR reporting trends over time. The completeness score of each report is calculated automatically. Therefore, a limitation of this study is that it could not be assessed, which specific information was missing on the reports submitted by different HCPs categories. This study used data downloaded from the VigiBase ® , and it does not indicate the sender of the report (i.e. MAHs, clinical trials, HCPs or non-HCPs) or the report reporting method (e.g. Med Safety App, e-reporting or paper form). The authors were, therefore, unable to assess the completeness by the sender of the reports. The preferred method to report could also not be determined from the data. A more in-depth analysis could be performed to distinguish the proportion of missing information according to sender and primary reporter and to compare the completeness of ADR reports between MAHs and HCPs.

Conclusion
This study's analysis indicates that the completeness of ADR reports submitted to SAHPRA is low and requires immediate intervention to improve the quality of reports. It was found that the completeness of reports submitted by consumers is comparable to those submitted by HCPs, which confirms that consumers have a potential role to play within PV. It is therefore imperative that consumer reporting is encouraged, explored and supported in the country. Multiple measures to improve ADR reporting and quality are recommended, particularly for HCPs. These may include using digital reporting tools or EHRS and increasing awareness for HCPs and the public on ADR reporting and reporting quality. The inclusion of a section in the PIL explaining how and where to submit an ADR is also recommended.